A Robot Learning to Play a Mobile Game Under Unknown Dynamics

With the advance in robotic hardware and intelligent software, humanoid robot could play an important role in various fields including service for human assistance and heavy job for hazardous industry. Under unknown dynamics operating smart devices with a humanoid robot is a even more challenging task because a robot needs to learn both swipe actions and complex state transitions inside the smart devices in a long time horizon. Recent advances in task learning enable humanoid robots to conduct dexterous manipulation tasks such as grasping objects and assembling parts of furniture. In this paper, we explore another step further toward building a human-like robot by introducing an architecture which enables humanoid robots to learn operating smart devices requiring complex tasks. We implement our learning architecture in the Baxter Research Robot and experimentally demonstrate that the robot with our architecture could play a challenging mobile game, the 2048 game, as accurate as in a simulated environment.